new: add possibility to retrieve details about rules with filter_details_visitor #2544

loresuso · 2023-05-12T10:25:46Z

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

/kind release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area tests

/area proposals

/area CI

What this PR does / why we need it:
This PR introduces a specialized visitor that can be used to construct a normalized form of a rule, containing all the macros, lists, operators, and fields it uses in JSON format. The functionality is embedded in the -l/-L flags and can be triggered when the output is set to JSON.
This functionality can be used later in CI tools for rules to e.g understand how to automatically handle and increase rules version.

Example:

$ sudo ./userspace/falco/falco -c ../falco.yaml -l "Launch Privileged Container" -o "json_output=true" | jq 
Fri May 12 10:23:13 2023: Falco version: 0.34.1-194+f5c7574 (x86_64)
Fri May 12 10:23:13 2023: Falco initialized with configuration file: ../falco.yaml
Fri May 12 10:23:13 2023: Loading rules from file /etc/falco/falco_rules.yaml
Fri May 12 10:23:13 2023: Loading rules from file /etc/falco/falco_rules.local.yaml
{
  "conditionFields": [
    "evt.num",
    "container.id",
    "container.privileged",
    "container.image.repository",
    "proc.vpid",
    "evt.dir",
    "evt.type"
  ],
  "description": "Detect the initial process started in a privileged container. Exceptions are made for known trusted images.",
  "exceptionFields": [],
  "lists": [
    "redhat_io_images_privileged",
    "falco_privileged_images",
    "trusted_images"
  ],
  "macros": [
    "redhat_image",
    "aws_eks_core_images",
    "user_trusted_containers",
    "allowed_openshift_registry_root",
    "openshift_image",
    "never_true",
    "falco_privileged_containers",
    "user_privileged_containers",
    "container",
    "spawned_process",
    "allowed_aws_ecr_registry_root_for_eks",
    "container_started"
  ],
  "name": "Launch Privileged Container",
  "operators": [
    "endswith",
    "startswith",
    "in",
    "!=",
    "="
  ],
  "outputFields": [
    "user.name",
    "user.loginuid",
    "proc.cmdline",
    "proc.pid",
    "container.name",
    "container.id",
    "container.image.repository",
    "container.image.tag"
  ],
  "priority": "Informational"
}

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

update: get rules details with `-l` or `-L` flags when json output format is specified

loresuso · 2023-05-12T10:29:50Z

cc @jasondellaluce

FedeDP · 2023-05-12T10:34:38Z

This is super cool and highly requested! Thank you @loresuso !

jasondellaluce · 2023-05-12T18:24:52Z

Links to #2372 and potentially fixes it, if we just add the information about the evt.types matched by each rule.

loresuso · 2023-05-15T12:53:17Z

Last two commits address the following:

retrieving information about enabled rules
information about lists and macros when the global -L option is specified

This will allow to check for instance (check https://github.com/falcosecurity/rules/blob/main/RELEASE.md for further details):

if a rule used to be disabled and then enabled, patch number of version of the rule must be increased
adding, removing elements from lists, same as above

Currently working on adding evt.type information

userspace/engine/rule_loader_compiler.cpp

Andreagit97

Left some minor comments, LGTM!

userspace/engine/falco_engine.cpp

Andreagit97 · 2023-05-16T17:37:40Z

userspace/falco/app/actions/load_rules_files.cpp

@@ -126,13 +126,13 @@ falco::app::run_result falco::app::actions::load_rules_files(falco::app::state&

 	if (s.options.describe_all_rules)


I would initialize both describe_all_rules and describe_rule variables here

options::options() : event_buffer_format(sinsp_evt::PF_NORMAL), gvisor_config(""), list_fields(false), list_plugins(false), list_syscall_events(false), markdown(false), modern_bpf(false) { }

They should already have their default I guess in options.cpp

userspace/engine/filter_details_resolver.h

leogr · 2023-05-18T13:20:24Z

/milestone 0.35.0

userspace/engine/filter_details_resolver.cpp

userspace/engine/falco_engine.cpp

jasondellaluce

Looks good Lore:

Left you a simple comment on that thing about exceptions and list of operators
Left you suggestions to cover corner cases in the macro/list visitor search
Once all's done, we'll need to rebase on top of update!: bump libs version, and support latest plugin features, add --nodriver option #2552, that should bump the libs and hopefully fix the broken tests

userspace/engine/filter_details_resolver.cpp

userspace/engine/falco_engine.cpp

Signed-off-by: Lorenzo Susini <[email protected]>

…etails when requested Signed-off-by: Lorenzo Susini <[email protected]>

Signed-off-by: Lorenzo Susini <[email protected]>

loresuso · 2023-05-19T11:37:54Z

Just rebased and everything's green, I think we are there now!

… getting details of all rules Signed-off-by: Lorenzo Susini <[email protected]>

Signed-off-by: Lorenzo Susini <[email protected]>

…ption Signed-off-by: Lorenzo Susini <[email protected]>

Signed-off-by: Lorenzo Susini <[email protected]>

… details Signed-off-by: Lorenzo Susini <[email protected]>

Signed-off-by: Lorenzo Susini <[email protected]>

- avoiding inspector to be allocated for each rule - use two boolean values for expecting macros and lists - move items of lists alongside name, under info - use snake case for json output, like we do for e.g alerts - correctly retrieve evt names - consider two levels of lists for exception operators Signed-off-by: Lorenzo Susini <[email protected]>

jasondellaluce

/approve

Thanks! This is great!

poiana · 2023-05-19T12:31:38Z

LGTM label has been added.

Git tree hash: 76ba42e28e821ab6c89ef3774628d821c834d360

Andreagit97

/approve

poiana · 2023-05-19T13:55:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Andreagit97, jasondellaluce, loresuso

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Andreagit97,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

poiana added release-note kind/feature area/engine dco-signoff: no size/XL labels May 12, 2023

poiana requested review from FedeDP and Kaizhe May 12, 2023 10:26

loresuso force-pushed the filter-details-visitor branch from 62bfedf to 0004597 Compare May 12, 2023 10:27

poiana added dco-signoff: yes and removed dco-signoff: no labels May 12, 2023

loresuso force-pushed the filter-details-visitor branch from 0004597 to 2527699 Compare May 12, 2023 10:32

loresuso force-pushed the filter-details-visitor branch from 3b1ff62 to e713ae0 Compare May 12, 2023 10:59

loresuso force-pushed the filter-details-visitor branch from c803516 to b9c8ac1 Compare May 16, 2023 09:35

loresuso commented May 16, 2023

View reviewed changes

userspace/engine/rule_loader_compiler.cpp Outdated Show resolved Hide resolved

loresuso force-pushed the filter-details-visitor branch 3 times, most recently from 9b2d14d to eabd030 Compare May 16, 2023 12:44

Andreagit97 reviewed May 16, 2023

View reviewed changes

userspace/engine/filter_details_resolver.h Show resolved Hide resolved

loresuso force-pushed the filter-details-visitor branch from ed9b102 to b886a00 Compare May 18, 2023 09:12

poiana added size/XXL and removed size/XL labels May 18, 2023

loresuso force-pushed the filter-details-visitor branch from b886a00 to e58a4a8 Compare May 18, 2023 09:39

poiana added this to the 0.35.0 milestone May 18, 2023

jasondellaluce reviewed May 18, 2023

View reviewed changes

Andreagit97 mentioned this pull request May 18, 2023

Log rule expansion under log_level=debug upon startup #1814

Closed

jasondellaluce reviewed May 19, 2023

View reviewed changes

loresuso added 3 commits May 19, 2023 10:35

new(userspace/engine): introduce new class to get details about rules

409a506

Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): let describe_rule function print out json d…

6d0dcac

…etails when requested Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): correctly use describe rule based on config

f3952b9

Signed-off-by: Lorenzo Susini <[email protected]>

loresuso force-pushed the filter-details-visitor branch from 5178406 to d97d862 Compare May 19, 2023 10:38

loresuso added 7 commits May 19, 2023 12:16

update(userspace/engine): avoid solving macros AST at each cycle when…

12f21dc

… getting details of all rules Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): add enabled information to json output

0f0d34a

Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): add info about all macros and lists in -L o…

053c17d

…ption Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): add event codes to json output

f424b14

Signed-off-by: Lorenzo Susini <[email protected]>

update(userspace/engine): better modularize the code for getting json…

3117700

… details Signed-off-by: Lorenzo Susini <[email protected]>

test(unit_tests/engine): test filter_details_resolver class

af111ea

Signed-off-by: Lorenzo Susini <[email protected]>

loresuso force-pushed the filter-details-visitor branch from d97d862 to f662353 Compare May 19, 2023 12:16

jasondellaluce approved these changes May 19, 2023

View reviewed changes

poiana assigned jasondellaluce May 19, 2023

poiana added the lgtm label May 19, 2023

poiana added the approved label May 19, 2023

jasondellaluce mentioned this pull request May 19, 2023

[New Feature]: Dry run option that outputs the syscalls used by rules #2372

Closed

Andreagit97 approved these changes May 19, 2023

View reviewed changes

poiana assigned Andreagit97 May 19, 2023

poiana merged commit e47ece4 into falcosecurity:master May 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new: add possibility to retrieve details about rules with filter_details_visitor #2544

new: add possibility to retrieve details about rules with filter_details_visitor #2544

loresuso commented May 12, 2023

loresuso commented May 12, 2023

FedeDP commented May 12, 2023

jasondellaluce commented May 12, 2023

loresuso commented May 15, 2023 •

edited

Loading

Andreagit97 left a comment

Andreagit97 May 16, 2023

loresuso May 18, 2023

leogr commented May 18, 2023

jasondellaluce left a comment

loresuso commented May 19, 2023 •

edited

Loading

jasondellaluce left a comment

poiana commented May 19, 2023

Andreagit97 left a comment

poiana commented May 19, 2023

		@@ -126,13 +126,13 @@ falco::app::run_result falco::app::actions::load_rules_files(falco::app::state&

		if (s.options.describe_all_rules)

new: add possibility to retrieve details about rules with filter_details_visitor #2544

new: add possibility to retrieve details about rules with filter_details_visitor #2544

Conversation

loresuso commented May 12, 2023

loresuso commented May 12, 2023

FedeDP commented May 12, 2023

jasondellaluce commented May 12, 2023

loresuso commented May 15, 2023 • edited Loading

Andreagit97 left a comment

Choose a reason for hiding this comment

Andreagit97 May 16, 2023

Choose a reason for hiding this comment

loresuso May 18, 2023

Choose a reason for hiding this comment

leogr commented May 18, 2023

jasondellaluce left a comment

Choose a reason for hiding this comment

loresuso commented May 19, 2023 • edited Loading

jasondellaluce left a comment

Choose a reason for hiding this comment

poiana commented May 19, 2023

Andreagit97 left a comment

Choose a reason for hiding this comment

poiana commented May 19, 2023

loresuso commented May 15, 2023 •

edited

Loading

loresuso commented May 19, 2023 •

edited

Loading